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The  paper  addresses  the  question  of  how  the  Inspector  General  of  the 


I Air  Force  Systems  Command  and  his  personnel  can  more  effectively  and  use- 

I • 

[ I fully  inspect  and  evaluate  research  organizations.  Based  upon  an  examina- 

f j _ tion  of  pertinent  regulations,  directives  and  the  open  literature,  and  based 

I - upon  interviews  with  knowledgeable  personnel,  the  roles  and  missions  of  both 

1 j the  Air  Force  laboratories  and  the  AFSC  inspectors  are  analyzed.  This  ana- 

I I lysis  is  then  used  to  discuss  where  current  practices  should  be  modified  in 

order  to  improve  the  meaningful  ness  of  inspections  of  research  organizations. 
Specific  attention  is  given  to  the  interface  between  the  inspectors  and  the 
staff  of  the  Director  of  Science  and  Technology.  The  final  part  of  the 


paper  deals  with  aspects  of  the  laboratory-system  mission  which  should  weigh 
more  heavily  in  the  overall  evaluation  of  a research  organization  than  they 
do  now. 
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The  management  and  conduct  of  research  should  be  a matter  of  concern 
to  all  Air  Force  personnel  involved  in  systems  acquisition.  The  reasons 
for  such  concern  do  not  lie  in  the  amount  of  money  involved,  though  it  is 
not  insignificant,  nor  in  the  numbers  of  people  involved.  Military  and 
civilian  writers  in  the  field  of  research  management  have  cogently  assessed 
the  importance  of  research  efforts.  [10»^2,13,18]'*’  Martino  and  Stephenson 
perhaps  express  it  best:  " ...  the  future  Air  Force  will  be  determined  by 
today's  research;  it  may  be  captive  of  a poorly  chosen  research  program  or 
beneficiary  of  a well  chosen  investigation."  To  well  chosen,  I would  only 
add  well  managed  and  well  conducted. 

It  is  appropriate  at  this  point  to  define  what  aspects  of  research  I 
am  talking  about.  Since  the  terms  basic  and  applied  research  are  subject 
to  much  interpretation,  let  me  use  Glennan's  definition  that  research  is 
that  activity  which  includes  "all  effort  directed  toward  increased  knowledge 
of  natural  phenomena  and  environment  and  toward  solutions  to  problems  in 
physical,  behavioral  and  social  sciences  having  no  clear,  direct  military 
application."  Into  this  definition  we  can  fit  nearly  all  of  the  basic 
research  (6.1)  and  some  of  the  exploratory  development  (6.2)  activities  of 
the  various  Air  Force  laboratories.  (We  could  also  include  research  per- 
formed by  other  organizations  such  as  the  USAF  Academy.)  Since  these  lab- 
oratories typically  have  broader  charters  than  implied  by  this  definition 
of  research,  I will  in  large  measure  restrict  the  considerations  of  this 
essay  to  their  research  and  research-related  activities. 

^ References  will  be  designated  by  a superscript  number  designating  the 
number  of  the  reference  in  the  List  of  References. 
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A separate  activity,  quite  apart  from  research,  that  is  of  special 
concern  to  military  commanders  is  inspection.  This  activity  is  an  impli- 
cit and  natural  function  of  command.  Keefe  notes  that  there  is  a basic 
conflict  in  the  Air  Force  between  the  freedom  of  science  and  the  control 
of  scientific  efforts  in  the  interest  of  the  Air  Force,  An  extreme 

position  on  the  side  of  scientific  freedom  would  deny  the  efficacy  of  in- 
spection (at  least  in  the  sense  of  a compliance  inspection)  in  the  research 
environment,  an  environment  characterized  in  [14]  as  one  of  extreme  uncer- 
tainty in  the  outcome  of  any  task.  The  opposite  position  is  reflected  by 
Dean,  who  notes  that  "at  both  national  and  corporate  levels  the  point  has 
been  reached  where  R&D  is  conceived  as  a production  process  subject  to 
planning,  control,  and  optimization."  Though  truly  not  a production 

process,  R&D  must  be  managed  and  the  inspection  process  is  a part  of  the 
commander's  management  information  system.  Thus  it  is  necessary  that 
there  be  inspection  of  research  activities  as  well  as  operational  activites. 

The  process  by  which  inspection  is  carried  out  has  been  subject  to  cri- 
ticism. Though  not  specifically  addressed  to  the  inspection  of  research 
activities,  recent  reports  have  professed  that  the  time  used  to  prepare 
could  be  more  productively  used  in  improving  internal  efficiency,  that 
the  system  is  a detriment  to  effective  operations,  and  that  there  has  been 
a failure  to  innovate  in  the  inspection  process  while  the  rest  of  Air  Force 
management  has  experimented  with  new  techniques  such  as  participative  manage- 
ment and  management  by  objectives.  This  last  charge  is  not  entirely 
correct,  for  the  Air  Force  Systems  Command  Inspector  General  (AFSC/IG)  has 
introduced  a number  of  important  innovations  (some  of  which  will  be  mentioned 
later).  However,  he  has  not  fundamentally  altered  the  manner  in  which  an  in- 
spection is  conducted  - how  the  inspector  interacts  with  the  inspectee  or 
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with  the  cognizant  staff  agency.  That  interaction  has  tended  to  be  adver- 
sarial in  nature. 

Despite  statements  by  staff  personnel  that  the  IG  should  be  looked 
upon  as  an  aid  to  effective  management  rather  than  an  adversary, 
and  despite  IG  operating  instructions  that  state  that  commanders  should 
welcome  inspections  as  direct  contributions  toward  improving  their  oper- 
ations rather  than  worry  about  surviving  an  inspection,  ^31]  people  are 
apprehensive  and  defensive  about  inspections.  Despite  the  statement  that 
"there  should  be  no  indication  of  'finger  pointing'  during  the  interview, 
exit  briefing,  or  report  writing  phases,"  current  inspectors  recog- 
nize that  fault  finding  (of  either  individuals  or  systems  or  both)  is  in- 

r22lt 

herent  in  an  inspection  finding.  A former  deputy  IG  even  states 

that  there  is  some  intimidation,  perceived  by  the  inspectee,  in  the  nature 
of  an  IG  report.  With  the  continued  deemphasis  of  strict  compliance 

inspections,  the  subjective  judgements  of  inspectors  heighten  the  friction 
in  the  process  though  they  also  increase  the  dialogue  between  inspector  and 
inspectee.  There  is  staff  concern  over  disclosing  Staff  Assistance 

Visit  reports  to  the  IG  for  fear  of  losing  the  confidence  of  the  subordinate 
organizations  they  deal  with  if  that  disclosure  might  lead  to  an  IG  finding. 

In  fact,  AFSC/DL  has  been  excused  from  the  requirement  to  provide  such  re- 
ports to  the  AFSC/IG.  Hartung-Schuster  notes  that  "it  could  be  said  that 

the  inspection  system  is  a hindrance  to  solving  problems  because  it  could 
cause  subordinate  commanders  to  conceal  problems  rather  than  expose  problem 
areas." 

^ The  need  for  fault-finding  is  based  on  the  requirement  that  the  cause  of  a 
finding  be  explicitly  stated  in  the  inspection  report.  Further,  experience 
has  made  the  IG  feel  that  a statement  of  cause  is  necessary  information  for 
effecting  corrective  action.  I.22J 


Given,  then,  the  current  criticisms  of  the  inspection  process  and  given 
the  breadth  of  research,  to  which  there  is  no  "universally  applicable  method" 
of  appraisal  yet  which  must  be  subject  to  better  control  by  Air  Force 
commanders  than  ever  before,  this  paper  addresses  ways  in  which  inspection 
might  be  more  effectively  applied  to  research  organizations.  I will  begin 
by  discussing  the  respective  objectives  of  the  laboratory  system  and  of  the 
IG.  This  discussion  provides  insight  as  to  what  is  expected  of  each.  That 
insight  leads  one  to  a better  view  of  what  it  is  that  should  be  examined 
during  the  inspection  of  research  activities  and  of  appropriate  ways  to  go 
about  the  process.  Next  I examine  the  interface  between  the  research  activ- 
ities, especially  at  the  staff  level,  and  the  IG.  It  is  at  this  interface 
that  the  most  significant  benefits  can  be  derived  in  terms  of  improved 
effectiveness  of  the  inspection  process.  Finally,  I will  make  some  sugges- 
tions concerning  areas  of  inspection  emphasis  that  can  improve  the  meaning- 
fulness of  inspections  of  research  activities. 

The  central  issue  throughout  this  essay  is  what  do  we  require  of  an  in- 
spection of  a research  activity  and  how  do  we  go  about  getting  that  from  the 
inspection  process?  I do  not  even  pretend  to  provide  the  answer,  but  I hope 
that  I shall  stimulate  introspection  in  both  the  research  and  IG  communities 
that  will  eventually  provide  the  answers  that  the  Air  Force  needs. 

The  Laboratories 

As  quoted  in  [7],  AFR  80-3  defines  the  mission  of  the  in-house  labora- 
tory as  one  of  providing  "the  Air  Force  with  scientific,  engineering  and 
analytic  support  in  creating  new  weapons,  vehicles,  and  equipment  and  in 
developing  future  concepts."  AFSCR  80-23  states  that  "The  primary  mission 
of  the  AFSC  laboratories  is  to  maintain  a technological  base  by  conducting 
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effective  research,  exploratory,  and  advanced  development  programs  .... 
laboratories  are  specifically  tasked  to  provide  support  to  ..."  AFSC 
field  commands  that  are  responsible  for  acquisition  of  current  systems. 

These  are  general  statements  of  what  the  Air  Force  expects  from  its 
laboratories.  Perhaps  more  important  than  these  regulatory  statements  are 
the  roles  perceived  to  be  appropriate  for  the  laboratories.  These  percep- 
tions exist  within  and  without  the  research  community,  are  flexible  and 
varying,  and  provide  a better  basis  upon  which  to  judge  whether  or  not  the 
laboratories  are  fulfilling  the  role  the  Air  Force  wants  them  to  at  any 
given  time. 

Dr.  Currie,  former  Director  of  Defense  Research  and  Engineering,  has 
stated  that  "a  principal  reason  for  the  laboratories'  existence  is  the  per- 
ception that  our  system  development  and  acquisition  function  needs  a cadre 
of  people  with  a combination  of  'hands  on'  technical  expertise  and  intimate 
famil iarity  with  Service  problems  without  industrial  bias.  The  POD 
Laboratory  Utilization  Study  (LUS)  states  that  "...  material  acquisition 
support  ...  must  be  viewed  as  the  principle  raison  d'etre  of  the  PS&E  labora- 
tories and  the  ultimate  performance  measure  of  importance  in  assessing  the 
contributions  of  the  laboratories." 

Obviously,  the  advancement  of  the  state  of  the  art  in  technology  is  one 
of  the  primary  roles  for  the  laboratories.  Kalisch  states  that  the  "mission 
of  Air  Force  basic  research  is  to  create  opportunities  for  the  exploitation 
of  science."  Martino  and  Stephenson  say  that  it  "is  to  provide  a more 

relevant  knowledge  base  to  support  future  military  technology."  ^13] 
emphasis  in  each  case  is  on  pursuing  that  technology  which  is  not  of  in- 
terest outside  the  military  community. 
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Another  role  perceived  by  Currie  and  many  others,  is  to  make  the  serv- 
ices "super  educated  buyers".  In  this  vein,  the  Air  Force’s  laboratory 
utilization  study  group  noted  that  the  laboratory  complex  must  be  able  to 
"interpret  the  weapons  system  significance  of  changing  technology  to  the 
service  users  ...  [and]  then  interpret  the  service  needs  to  the  industrial 
base." 

Training  is  an  important  laboratory  function  and  one  which  Col  Sigethy, 
Director  of  Plans  and  Operations  for  the  Air  Force  Office  of  Scientific 
Research  (AFOSR),  feels  to  be  among  the  more  important,  'nat  the 

laboratories  play  a valuable  part  in  the  education  and  trainiiig  of  the 
young  scientists  and  engineers  who  will  develop  into  the  senior  managers  of 
the  acquisition  process  has  also  been  noted  by  the  Chief  Scientist  and 
Director  of  Plans  and  Programs  for  the  Director  of  Science  and  Technology 
(AFSC/DL).  [7] 

Outside  of  DOD,  the  Office  of  management  and  Budget  has  stated  what  it 
believes  to  be  a valid  role  for  the  laboratories:  "Government  laboratories 
should  be  considered  for  participation  in  agency  mission  analysis,  evalua- 
tion of  alternative  system  design  concepts,  and  support  of  all  development, 
test,  and  evaluation  efforts."  This  has  led  to  the  specification  of 

a Technology  Assessment  Annex  to  be  attached  to  Decision  Coordinating  Papc  s 
(DCP)  that  are  reviewed  by  the  Defense  Systems  Acquisition  Review  Council 
(DSARC). 

Some  other  functions  the  laboratories  should  perform  in  the  pursuit  of 
their  basic  objectives  include:  provision  of  engineering  support  to  field- 
ed systems,  provision  of  a technological  corporate  memory,  provision  of 
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scientific  intelligence  relative  to  potential  enemies'  systems,  and  pro- 
vision of  a rapid-response  capability  to  solve  immediate  problems  in 
technology. 

One  final  point,  noted  by  Sigethy,  is  that  the  existence  of  a tech- 
nological community  within  the  Air  Force  sets  a tone  and  tenor  to  the 
entire  thought  process  of  the  service  that  enhances  its  ability  to  function 
effectively  in  the  highly  complex  technological  environment  of  the  present. 

The  more  competent  that  community  is,  the  more  pervasive  its  influence  will 
be. 

Briefly  summarizing  at  this  point,  I ask  that  you  appreciate  the  wide- 
ranging  perspectives  that  exist  relative  to  what  the  laboratories  should  be 
doing  for  the  Air  Force.  These  perspectives  include  technology  advancement, 
personnel  development,  technical  intelligence,  and  system  acquisition  support. 

The  Inspector  General 

It  is  appropriate  to  begin  the  discussion  of  the  IG,  as  was  done  for 
the  laboratories,  by  considering  the  governing  regulations  and  directives 
and  then  looking  at  how  the  perceptions  of  those  involved  affect  the  actual 
inspection  process.  AFR  123-1  states  that  the  objectives  of  the  inspection 
system  are  to  provide  commanders  with: 

a.  A capability  to  maintain  continuing  surveillance 
over  the  status  of  readiness  within  the  commands. 

b.  A measure  of  effectiveness  and  efficiency  of 
management  systems. 

c.  A management  technique  to  identify,  assess,  and 
resolve  significant  problems  and  to  recognize 
exceptional  managers  and  management  practices. 

d.  An  evaluation  of  the  adequacy  of  safety  and 
occupational  health  programs  to  include  appli- 
cable provisions  of  . . . OSHA  ... 


e.  Factual  information  on  which  to  base  action  if 
a management  system  is  not  achieving  maximum 
effectiveness. 

IGOI  123-1  declares  the  purpose  of  AFSC/IG  inspections  to  be  the  evalu- 
ation of  the  ability  of  AFSC  activities  to  effectively,  safely,  and  econo- 
mically perform  their  missions  and  functions;  to  recognize  the  existence 
of  problems  which  impede  mission  accompishment;  and  to  provide  assessments 
for  the  Commander,  AFSC  of  how  well  command  organizations  are  accomplishing 
their  assigned  missions, 

Within  these  words  are  the  seeds  of  much  of  the  controversy  that 
surrounds  the  inspection  process.  The  reasons  for  the  controversy  are 
the  dichotomous  positions  that  can  be  assumed  based  on  the  formal  direc- 
tives. Justification  can  be  found  to  support  inspection  as  a "fact- 
finding mission,"  to  support  compliance  inspections  (as  well  as  to 
reject  them  ^28]^  and  to  support  subjective  management  inspections.^^^^ 

Additional  controversy  centers  on  the  adversarial  and  fault-finding  nature 
of  inspections  which  was  discussed  earlier.  The  disparity  of  perceptions 
among  those  involved  in  the  inspection  process  (both  inspectors  and  in- 
spectees)  does  little  to  dampen  the  controversy. 

There  is  one  aspect  of  inspection  about  which  there  appears  to  be  a 
concensus.  While  the  how  of  inspection  is  disputed,  the  why  of  inspection 
is  generally  agreed  to  be  to  assist  the  commander  in  fulfilling  his  manage- 
ment responsibility  for  control  of  his  organization  by  functioning  as  part 
of  the  commander's  management  information  system.  [9»22,24,25]  jf, 
capacity,  the  IG  must  reflect  the  commander's  concerns,^^^^  and  the  thrust 
of  an  inspection  must  be  focused  in  the  direction  that  the  commander  desires.  [23] 
The  information  that  the  IG  provides  the  commander  must  be  factual  in  basis 
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if  command  decisions  are  to  be  reliably  made. 

Once  past  the  factual,  objective  aspects  of  inspection,  one  enters  the 
subjective  arena  in  which  much  of  the  controversy  exists.  At  the  heart  of 
the  issue  is  the  question  of  how  much  compliance  should  be  emphasized  ver- 
sjs  how  much  management  effectiveness  should  be  emphasized.  Robertson  de- 
fines a compliance  inspection  as  one  which  "determines  conformity  with 
formal,  published,  official  requirements,"  and  a management  inspection  as 
one  which  "determines  the  effectiveness  with  which  managers  are  effecting 
their  responsibilities."  The  argument  as  to  which  approach  is  proper 

predates  the  Air  Force  itself,  and  Trace  appropriately  notes  that  "the  USAF 
Inspection  System  was  born  amid  controversy." 

Arguing  against  excessive  compliance  requirements,  Sigethy  expresses 
the  interesting  concept  that  such  an  inspection  is  a suboptimization  pro- 
cess. Strict  adherence  to  individual  regulations  and  directives  in- 

sures the  individual  effectiveness  of  each,  but  only  at  the  expense  of  the 
overall  effectiveness  of  an  organization.  It  is  the  overall  effectiveness 
of  an  organization  however,  that  is  ultimately  important. 

The  trend  today  is  away  from  compliance  inspections.  AFR  123-1 

states  that  "purely  compliance  oriented  inspections  are  to  be  avoided" 
avoided"  and  notes  that  compliance  problems  are  to  be  avoided  unless  they 
are  indicative  of  an  underlying  management  problem.  ^28]  j|^g  afsC/IG 
operating  instruction  flatly  states  that  "there  is  nothing  wrong  with  in- 
cluding judgement  findings  in  a report." 

Management  inspections  are  more  difficult  to  accomplish  than  compli- 
ance inspections  because  of  the  subjectivity  involved.  They  also  increase 
the  friction  between  inspectors  and  inspectees  as  was  mentioned  earlier. 
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Some  of  the  AFSC/DL  staff  are  concerned  Test  this  trend  go  too  far  and 
feel  that  there  may  already  be  too  much  subjectivity  entering  the  inspec- 
tion process.  Monts  notes  that  "the  IG  must  keep  parochialism  under 

control  to  prevent  inspectors  from  using  their  position  to  arbitrarily 
make  management  reconmendations  in  their  findings  based  on  personal  pre- 
conceived ideas  that  directly  conflict  with  the  policy  and  background  of 
the  responsible  staff."  If  carried  to  an  extreme,  one  can  visualize 

the  management  judgement  of  the  IG,  derived  from  the  short  period  of  exam- 
ination available  to  him,  overriding  the  judgement  of  field  conmanders  and 
staff  personnel,  who  bear  the  responsibility  for  the  organizations  involved. 
This  does  not  deny  the  necessity  for  the  IG  to  examine  the  validity  of  poli- 
cies that  are  in  effect.  The  AFSC/IG  is  aware  of  this  adverse  potential, 
and  he  instructs  his  inspectors  to  exercise  discretion  in  the  application  of 
management  principles  based  upon  the  situation  at  hand.  Emphasis  is  to  be 
placed  on  the  "results  and  economy  of  obtaining  those  results." 

I have  now  separately  discussed  the  roles  and  objectives  of  the  labora- 
tory and  inspection  systems.  From  these  discussions,  it  is  clear  that  in 
his  search  to  provide  the  factual  information  upon  which  the  Commander,  AFSC 
can  base  action  to  assure  effective  and  efficient  mission  accomplishment, 
the  AFSC/IG  needs  to  fully  understand  the  entire  range  of  the  laboratories' 
roles  and  he  must  subjectively  examine  many  of  the  activities  in  which  the 
laboratories  engage.  The  next  two  sections  of  this  paper  address,  respec- 
tively, how  this  understanding  can  be  better  assured  and  a few  important 
activities  that  should  play  larger  parts  in  the  overall  evaluation  of  a 


The  Interface  - A Lack  of  Cotnmuni cation 


If  the  inspection  process  is  to  function  in  a worthwhile  manner,  there 
must  be  a two-way  flow  of  information  and  understanding  across  the  inter- 
face between  the  research  activity  and  the  inspector. This  communication 
flow  must  take  place  before,  during  and  after  the  actual  inspection.  Both 
parties  must  seek  to  close  the  communication  loop  around  the  interface  and 
attempt  to  make  the  other  smarter  about  his  own  objectives  and  requirements. 
Today  that  loop  is  not  adequately  closed;  in  Sigethy's  words,  communication 
between  the  research  activities  and  the  IG  is  incomplete.  Blame  for 

this  problem  can  be  ascribed  to  both  sides.  Research  activities  have  failed 
to  make  clear  to  the  IG  those  matters  that  are  truly  important  to  the  success- 
ful fulfillment  of  their  prescribed  and  perceived  roles.  Nor  have  they  pro- 
vided the  rationale  concerning  the  formal  documentation  that  will  provide 
the  IG  the  insight  to  make  the  proper  subjective  judgements  during  an  inspec- 
tion. The  inspector,  in  turn,  has  not  asked  the  penetrating  questions  that 
must  be  asked  before  such  insight  can  be  gained. 

The  relatively  unstructured  nature  of  the  research  environment  as  com- 
pared to  an  operational  environment  only  serves  to  exacerbate  the  problem 
of  communicating  these  concepts.  The  inherent  uniqueness  of  each  research 
facility  further  compounds  the  difficulty.  Though  the  inspector  does  not 
evaluate  technical  competence  per  se,  he  does  evaluate  the  effectiveness 
of  management  in  obtaining  technical  ends.  He  must,  therefore,  be  knowledge- 
able to  some  moderate  degree  about  those  technical  ends,  for  management  must 

^ I include  both  the  laboratories  and  the  AFSC/DL  staff  in  the  term  research 
activity  and  use  "inspector"  to  mean  the  inspection  team  rather  than  a 
single  individual. 
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tailor  its  activities  to  them.  For  example,  an  item  in  the  AFSC/IG  Check- 
list asks  whether  the  laboratory  Technical  Planning  Objectives  contain  ade- 
quate descriptions  of  projects.  Technical  comprehension  is  required  to 

make  such  an  evaluation.  The  current  IG  approach  to  providing  knowledgeable 
inspectors  is  to  form  IG  teams  comprised  "of  personnel  whose  working  experi- 
ence has  been  in  the  field  they  are  inspecting."  Within  R&D  however,  the 
disciplines  are  exceptionally  diverse,  thereby  limiting  the  carryover  value 
of  technical  experience  and  accentuating  the  need  for  full  understanding  of 
organizational  objectives  to  be  conveyed  at  the  interface  between  the  research 
activity  and  the  IG. 

The  most  appropriate  time  for  this  educational  process  to  take  place 
is  during  the  preparations  prior  to  an  inspection.  Detailed  preparations 
essentially  begin  when  the  inspection  subteam  chiefs  assign  functional 
area  responsibilities  to  the  individual  team  members.  The  inspectors 

then  formulate  their  personal  checklists  for  the  inspection  based  upon  all 
the  information  that  they  can  gather.  Their  basic  charge  is  to  "familiar- 
ize themselves  in  detail  with  the  organization  and  key  personnel  of  units 
to  be  visited,  the  breakout  of  responsibilities  within  their  particular 
areas  of  interest,  pertinent  findings  from  previous  efforts  which  may  be 
applicable  to  their  areas  of  interest,  and  current  problems  as  reported 
in  the  TIG  Brief  and  any  other  available  documents."  They  are  also 


tasked  to  contact  major  headquarters  staff  offices  on  items  selected  or 
proposed  for  inspection  and  for  inputs  from  the  staff  as  to  additional 


areas  which  should  receive  inspection  emphasis. 


Within  this  guidance. 


each  inspector  defines  his  own  approach  to  meet  the  specific  responsibili- 
ties and  objectives  worked  out  by  the  team  as  a whole. 
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The  key  to  the  entire  inspection  is  the  checklist  that  the  individual 
inspector  makes  up.  The  AFSC  Inspection  Checklists  are  formulated  on  an 
organizational  basis;  i.e.,  there  are  parts  for  program  offices,  for 
ranges  and  test  centers,  for  laboratories,  etc.  The  part  for  laboratories 
is  broken  into  seven  sections  dealing  with  organization  and  management, 
planning,  work  unit  management,  engineering  services/interfaces,  R&D  pro- 
curement, ADPE/Software , and  documentation.  These  basic  lists  "must  be 
tailored  and  supplemented  ...  as  required.  Each  inspector  must  include 
items  resulting  from  special  inspection  requirements,  directions  from  the 
inspection  charter,  current  command  interest  items,  and  functions  unique 
to  the  unit  to  be  inspected." 

Information  for  the  tailoring  comes  from  a number  of  sources.  Docu- 
mentary sources  may  include  regulations,  directives.  Staff  Assistance  Visit 
Reports,  Special  Subject  Regulations  (SSR),  Special  Interest  Items  (SII), 
Special  Checklist  Items  (SCI),  other  inspection  reports  and  the  TIG  brief. 
The  SI  Is  and  SCIs  are  usually  based  on  items  submitted  by  a staff  member 
who  desires  a close  examination  of  a certain  functional  area.  The  origi- 
nal request  to  the  staff  for  inputs  relative  to  a specific  inspection 
is  made  simultaneously  with  the  notice  of  inspection  that  is  given  to 
the  unit  involved.  Conferences  in  which  the  DL  staff  briefs  the  IG 

team  on  laboratory  technology  and  missions  are  often  held  as  much  as  60 
days  before  an  inspection.  [22,23] 

AFSC/DL  has  designated  an  office  (DLXB)  for  the  responsibility  to  coor- 
dinate the  interface  between  the  IG  and  the  staff  and  laboratories.  Parrish 
says  that  there  are  essentially  two  distinct  types  of  information  requested 
by  the  16.  The  first  concerns  identification  of  those  organizations  and 
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; * functional  areas  (such  as  the  Job  Order  Cost  Accounting  System)  that  are 

appropriate  for  inspection.  The  second  pertains  to  specification  of  im- 

i 

portant  items  to  be  checked  at  the  organization  selected  for  inspection.  | 

The  provision  of  the  information  involves  written  responses  and  personal  i 

} 

contact.  The  degree  of  personal  contact  depends  upon  the  individual  in-  i 

! spector,  for  it  is  typically  left  up  to  him  to  decide  what  information  he  j 

wants  from  the  staff.  Some  inspectors  spend  a lot  of  time  talking  with  i 

AFSC/DL  personnel  and  some  do  not  spend  much.  \ 

The  basis  for  an  effective  interchange  between  the  IG  and  the  research  | 

t 

, activity  clearly  exists,  but  it  is  not  used  to  the  advantage  it  should  be.  * 

f The  conversations  take  place,  the  letters  go  forth,  but  the  communication  | 

is  inadequate.  The  letters  that  AFSC/DL  sends  to  the  IG  identifying  specific  \ 

items  for  consideration  in  the  inspection  of  a unit  are  simply  listings  of  j 

i 

things  to  check.  They  typically  say,  for  example,  to  check  on  the  implemen-  i 

; I 

tation  of  a regulation,  directive,  or  procedure.  There  is  no  insight  pro-  \ 

vided  as  to  the  genesis  of  the  requirement,  nor  as  to  the  relative  importance  j 

of  various  aspects  of  the  requirement  or  the  relative  importance  among  re-  J 

quirements.  If  the  inspector  does  not  come  and  ask  for  clarification,  no  j 

• staff  attempt  is  made  to  provide  it.  ^23,24]  unfortunate  omis-  | 

, 1 

' ^ 

sion,  for  without  such  understanding,  the  perceived  roles  and  functions  ! 

are  unlikely  to  be  understood  and  a meaningful  evaluation  of  whether  an  or- 


ganization is  effectively  performing  its  mission  cannot  be  made. 

Clear  communication  of  the  technological  role  of  a given  laboratory 
has  not  been  accomplished  if  the  inspector  cannot  reliably  ascertain  who 
the  principal  users,  or  customers,  of  its  output  are.  Yet  the  inspector 
indicates  that  he  cannot  always  determine  beforehand  where  the  interfaces 


1 

( 


are,  who  the  customers  are.  (I  will  discuss  the  use  of  this  knowledge 

in  the  next  section.)  It  is  of  course  true  that  the  research  we  are  con- 
cerned with  here  will  typically  not  have  direct  military  application,  but 
if  management  cannot  determine  where  the  technology  outputs  of  a laboratory 
have  been  used,  with  whom  a laboratory  has  coupling  efforts,  for  whom  a 
laboratory  performs  consulting  or  other  support  efforts,  then  management  is 
not  doing  its  job.  If  it  does  know  these  things,  then  it  can  and  should 
communicate  them  and  their  importance  relative  to  the  overall  laboratory 
mission  to  the  16. 

The  inspector  is  at  fault  in  this  communication  failure  because  he 
does  not  ask  sufficiently  penetrating  questions  of  the  staff. 

When  he  receives  back  a list  of  things  to  be  checked  he  does  not  suffi- 
ciently probe  for  the  rationale  behind  the  requirement,  the  understanding 
of  what  is  important  and  what  is  not.  He  is  also  at  fault  for  restricting 
his  requests  for  information  to  the  AFSC/DL  staff.  Contact  with  groups 
such  as  the  Research  Objectives  Panel,  the  Research  Utilization  Council, 
or  the  research  directors  at  AFOSR  could  provide  additional  insight  and 
understanding  into  the  science  and  technology  being  pursued  and  what  might 

r??! 

be  required  to  manage  it,  but  they  are  not  routinely  querried. 

Admittedly,  finding  the  time  to  make  these  contacts  would  be  difficult. 

There  has  been  an  innovation  (begun  in  the  summer  of  1976)  by  the 
AFSC/IG  that  takes  a long  step  in  the  right  direction.  A staff  liaison 
officer,  designated  by  AFSC/DL,  accompanies  the  inspection  team  as  an  ad- 
visor. The  staff  representative  provides  advance  information  to  the  in- 
spection team  and  provides  real-time  constructive  inputs  to  the  inspection 
by  commenting  on  findings,  potential  causes  and  recommendations.  Not  only 
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, 

is  the  inspection  enhanced  by  this  approach,  but  the  friction  between  the 

staff  and  the  inspectors  is  reduced  by  the  improved  communication  between 

them.  Both  AFSC/DL  and  the  IG  have  been  very  satisfied  by  the  results  of 

[22  241 

this  effort  to  date.  ’ Presently,  AFSC/DL  leans  toward  selecting  an 
individual  from  the  directorate  having  primary  responsibility  for  the  organi- 
zation to  be  inspected  rather  than  an  individual  with  a more  general  perspec- 
tive from  plans  and  programs.  This  is  appropriate,  but  care  should  be 

taken  to  maximize  the  amount  of  input  the  staff  liaison  individual  receives 
from  the  other  AFSC/DL  staff  offices  having  responsibility  for  broad  func- 
tional areas.  This  need  is  analogous  to  the  need  to  provide  similar  under- 
standing to  the  IG. 

A different,  but  related,  approach  taken  by  the  IG  is  the  use  of  aug- 

mentees  for  the  inspection  team.  The  augmentee  is  a full-fledged  member 

of  the  inspection  team  provided  by  "HQ  AFSC  major  staff  offices  or  AFSC 

field  commands  and  laboratories."  The  augmentee  provides  technical 

expertise  that  can  be  used  to  look  for  duplication  of  technical  effort, 

problems  in  technical  management,  and  failures  or  oversights  in  the  techni- 

[221 

cal  direction  that  is  being  provided  by  management.  '■  More  than  being 
a generator  of  findings  (he  is  typically  less  productive  of  findings  than 
trained  inspectors  the  augmentee  is  a source  of  knowledge,  understand- 

ing and  perspective  that  enables  the  inspection  team  to  do  a better  job. 

If  the  inspection  team  recognizes  this  fact,  the  value  of  the  augmentee  will  | 

I 

be  high. 

I urge  greater  use  of  both  the  staff  liaison  officer  and  the  technical 
augmentee.  A possible  variation  of  the  use  of  a technical  augmentee  is  to 


16 


have  an  evaluation  of  the  laboratory  program  completed  in  a manner  similar 
to  that  by  which  independent  Research  and  Development  Reviews  are  accom- 
plished based  upon  written  brochures.  The  input  would  be  provided  to  the 
inspection  team  during  their  preparation  period.  Unfortunately  the  rewards 
to  the  augmentee  that  exist  because  he  now  is  able  to  personally  participate 
in  technical  exchanges  at  the  inspected  unit  are  denied  him  if  he  is  only  re- 
quested to  provide  a pre-inspection  evaluation.  Lack  of  motivation 

could  interfere  with  obtaining  useful  results.  An  alterntive  approach  in 
this  vein  would  be  to  have  an  augmentee  accompany  any  pre-inspection  visit 
by  AFSC/IG  personnel  to  the  organization  in  question. 

Finally,  consider  the  post  inspection  period.  Here  too  there  is  room 
for  improved  communication.  The  IG  operating  instruction  says  that  "it  is 
essential  that  DCSs  and  Chiefs  of  Special  Staff  Offices  be  familiar  with 
inspection  results  pertaining  to  their  functional  areas  of  responsibility." 
The  same  attempts  made  to  improve  pre-inspection  communication  need  to  be 
made  after  the  inspection. 

The  USAF/IG  has  an  analysis  system  that  can  provide  data  concerning 
common  Air  Force-wide  problems  arising  in  inspections.  There  have  been 
retrieval  problems  with  this  system,  so  the  AFSC/IG  is  developing  a de- 
ficiency analysis  system  that  will  be  particularly  applicable  to  AFSC.  The 
research  activities  must  make  full  use  of  this  feedback  mechanism.  Not  only 
will  it  help  provide  them  with  a clearer  picture  of  the  deficiencies  in  the 
field,  but  it  will  also  give  them  a fuller  understanding  of  current  IG  per- 
spectives and  objectives.  This,  in  turn,  will  allow  the  staff  to  provide 
better  information  and  understanding  to  the  IG  for  future  inspections. 


The  subject  of  this  section  has  been  the  need  to  improve  the  communi- 
cation link  between  the  research  activities  and  the  inspectors.  The  ob- 
jective is  valid  throughout  the  Air  Force,  but  is  especially  valid  in  the 
research  environment  where  even  similar  type  units  (e.g.,  laboratories) 
are  significantly  different.  The  interface  between  the  research  acitivities, 
especially  the  AFSC/DL  staff,  and  the  IG  must  be  used  to  transmit  under- 
standing and  perspective  to  both  parties  to  a greater  extent  than  now  occurs. 
Inspectors  must  seek  broader  input  to  and  fuller  understanding  of  those 
items  that  they  do  include  in  their  checklists.  The  research  activities 
must  take  the  initiative  in  providing  the  information  they  think  the  IG 
needs  to  have  in  order  to  perform  a valid  inspection.  The  current  initia- 
tive in  the  area  of  staff  liaison  officers  should  be  exploited  as  fully  as 
possible  and  increased  use  of  augmentees  should  be  considered. 

Areas  for  Inspection  Emphasis 

Having  examined  the  matter  of  improved  communication  at  the  inter- 
face between  the  research  activity  and  the  inspector,  I turn,  in  this  final 
section,  to  an  examination  of  some  of  the  specific  areas  of  importance  that 
are  relevant  to  the  evaluation  of  how  effectively  a research  activity  is 
fulfilling  its  role  in  the  Air  Force. 

The  perspective  brought  to  bear  in  selecting  the  areas  for  emphasis 
should  be  fully  cognizant  of  the  unique  nature  of  research  and  the  require- 
ments this  nature  places  on  management.  An  Army  policy  statement  reflects 
an  excellent  philosophy  from  which  to  begin:  "Management  will  emphasize  the 
emergence  and  application  of  new,  useful  and  bold  ideas,  rather  than  adminis- 

n 2] 

trative  smoothness  of  operations."  Reflecting  this  philosophy,  one  must 
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address  the  fulfillment  of  the  raison  d'etre  rather  than  the  details  of  the 


administrative  process  supporting  that  fulfillment.  In  fact,  a valid  inspec- 
tion item,  recognized  by  the  IG,  is  that  the  administrative  process  should 
not  be  an  overbearing  burden.  It  is  worthwhile  noting  that  virtually  the 
same  philosophy  is  cited  by  Dean  for  application  by  industry:  "The  final 
measure  of  system  effectiveness  is  maximum  profit  to  the  corporation  rather 

rsi 

than  blind  conformance  to  any  system."  ■' 

Much  discussion  can  be  found  in  the  literature  concerning  important 

considerations  in  the  evaluation  of  research  activities.  Frequently  cited 

is  the  resultant  impact  of  the  research  effort  on  company  operations 

or,  in  slightly  different  terms,  it's  "utility  or  value  ...  to  the  mission 

ri4l 

of  the  agency  or  society."  The  time  period  over  which  that  utility 

is  considered  is  important;  an  extended  period  of  time  should  be  examined. 

The  Committee  on  Federal  Laboratories  (COFL)  report  notes  that  "the  nature 
of  the  R&D  process  ...  mitigates  against  any  meaningful  measures  of  per- 
formance for  periods  as  short  as  a year." 

In  terms  of  measuring  the  contribution  of  basic  research,  Kalisch 
cites  four  important  considerations:  (1)  the  creation  of  new  fields  of  en- 
deavor; (2)  the  innovations  in  applied  science  that  result  from  an  effort^ 

(3)  the  USAF  relevancy  of  the  body  of  knowledge  within  a discipline,  and 

(4)  the  ability  of  science  to  relate  to  Air  Force  problems  and  the  Air 

Force's  understanding  of  science.  Dean  talks  about  the  contribution 

of  a research  project  in  only  slightly  different  terms.  He  states  that 
"project  evaluation  is  usually  based  on  (1)  an  overall  measure  of  corpo- 
rate performance  such  as  profits  ...;  (2)  a tabulation  of  relevant  project 

^ Project  Hindsight  was  an  unsuccessful  effort  to  formalize  the  tracking 
of  those  innovations. 
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factors  affecting  corporate  performance,  and  (3)  the  analysis  of  the  ways 

rsl 

in  which  these  factors  contribute  to  performance;  ..." 

In  accomplishing  the  research  ends  reflected  by  the  above  considera- 
tions, management  needs  to  be  flexible  enough  to  deal  with  short-term  prob- 
lems, steadily  and  patiently  pursue  long  term  objectives,  have  the  courage 
to  fight  for  the  programs  it  believes  are  necessary,  and  be  alert  to  happen- 
ings in  the  scientific  world  at  large.  Though  an  ability  to  pursue 

long  term  objectives  is  important,  rigid  adherance  to  plans  may  be  counter- 
productive as  well  as  impractical.  The  COFL  notes  that  an  important  con- 
sideration for  evaluating  management  efficiency  is  not  differences  between 
accomplishments  and  plans,  but  the  ability  to  explain  such  variances. 

A last  point  in  this  general  discussion  concerns  the  relations  between 
the  laboratories  and  the  users  of  their  output.  The  LUS  says  that  in  system 
development  by  program  managers  this  relationship  should  be  one  of  customer 
and  supplier.  Dean  states  that  an  R&D  program  in  industry  may  be  con- 

sidered a success  when  its  projects  are  being  handled  competently  and  the 
corporate  operating  divisions  are  satisfied  with  the  output. 

In  light  of  the  preceding,  a list  of  important  considerations  for  eval- 
uating a research  activity  might  enumerate  them  as  follows:  (1)  Is  the 
overall  mission  being  accomplished  by  effective  and  efficient  management? 

Are  the  outputs  or  products  of  a research  activity  useful  to  its  customers, 
in  sufficient  quantity  to  meet  their  needs,  and  is  technical  performance 
satisfactory?  (3)  Are  the  outputs  appropriate  to  the  organizations 's  mission 
and  do  they  contribute  to  its  accomplishment? 

There  are  many  quantifiable  measures  of  research  accomplishment  that 
reflect  upon  management  effectiveness  and  that  may  be  applicable  to  a greater 
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or  lesser  extent  in  any  given  circumstance.  Among  these  are  the  obvious 
things  such  as  numbers  of  technical  papers  and  reports  written;  numbers  of 
patents  received;  frequency  of  citation  of  published  works;  the  numbers  of 
creative  and  technically  competent  researchers  employed  and  the  ability  of 
the  organization  not  to  fragment  their  efforts;  ratings  received  through  a 
peer  review  process;  numbers  of  Technology  Needs  supported;  membership  by 
the  research  staff  in  professional  associations  and  their  participation  in 
advanced  education;  numbers  of  on-time  project  completions;  the  accuracy  of 
technical,  time  and  cost  estimates;  and  the  timeliness  and  adequacy  of  re- 
search dissemination.  H .2,5,12,13,14,17,18,19,21 ,23]  easily 

be  extended  but  one  must  recognize  that  quantitative  determinations  cannot 
be  used  as  final  measures.  They  are  only  useful  guides  and  their  trends  are 
likely  to  be  more  significant  than  their  absolute  magnitudes.  The  COFL 
further  cautions  that  many  of  these  items  such  as  numbers  of  patents,  or 
publications  and  frequency  of  citation  may  be  completely  inappropriate  in 
the  federal  laboratory  environment. 

There  are  also  measures  that  are  not  precisely  quantifiable,  yet  which 
do  provide  a strong  indication  of  the  degree  to  which  the  general  consider- 
ations discussed  earlier  in  this  section  are  satisfied.  I believe  that 
these  measures  do  not  receive  sufficient  emphasis  in  present  inspections. 

I will  discuss  a few  of  the  ones  I feel  to  be  more  important  in  the  remain- 
ing paragraphs.  Specifically,  I will  touch  on  customer  relations,  personnel 
management  and  development,  personnel  awareness,  and  project  evaluation. 

I have  mentioned  the  supplier-customer  relationship  previously  as  a 
general  criteria  for  laboratory  performance  and  as  a consideration  in  im- 
proved communication  between  the  research  activity  and  the  IG.  The  AFSC/IG 
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Checklist  Guide  mentions  the  interactions  of  a laboratory  with  the  users 
of  its  technology  only  briefly.  It  asks  what  the  magnitude  of  involvement 
with  product  divisions  is  and  what  coupling  has  been  implemented. 

As  Sigethy  indicates,  care  is  required  in  the  determination  of  who  the  cus- 
tomers are  but  once  they  are  identified,  it  is  important  to  determine 

whether  the  customers  are  satisfied.  The  LUS  states  that  laboratory  input 
should  be  provided  to  tiie  decision-making  process  up  through  DSARC.  This 
input  should  reflect  upon  the  "technical  merits  of  courses  of  action  pro- 
posed by  [the  laboratory's]  customers..."  It  is  equally  valid,  and 
especially  appropriate  during  inspection,  to  determine  the  degree  of  user 
satisfaction  with  the  laboratory.  One  indicator  of  such  satisfaction  is 
the  amount  of  consulting  that  laboratory  personnel  are  requested  to  do  to 
solve  problems  for  various  customers. 

A very  important  aspect  of  the  role  perceived  for  research  activities 
is  their  function  as  a training  ground  for  future  high-level  managers.  The 
LUS  specifically  mentions  the  use  of  laboratories  for  that  purpose.  Though 
SI  Is  at  times  emphasize  personnel  management,  there  is  an  absence  of  refer- 
ence to  this  activity  in  the  inspection  checklist.  Kellog  notes  that  "early 
identification  of  creative  scientific  personnel  ...  [is  among]  the  most  de- 
sireable  skills  to  be  sought  in  research  managers."  Management  should 

also  foster  development  of  people  by  providing  and  encouraging  the  use  of 
opportunities  to  upgrade  their  technical  and  management  skills.  A potential- 
ly valuable  consideration  is  where  people  go  when  they  leave  the  laboratory 


and  how  well  they  do  when  they  get  there.  A study  along  the  first  of  these 
lines  has  been  done  by  Capt  Sells  of  the  Director  of  Science  and  Technology 
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well  the  laboratories  develop  their  people  for  future  assignments.  (I  do 
not  intend  to  imply  that  the  IG  should  undertake  studies  such  as  this.  He 
should,  however,  evaluate  management  effectiveness  in  achieving  this  goal.) 

Closely  related  to  the  preceding  issue  is  the  development  of  awareness 
in  laboratory  personnel.  In  talking  about  the  maintenance  of  technical  ex- 
pertise as  a responsibility  of  the  laboratories,  the  LUS  goes  on  to  say  that 
"Operational  expertise  is  neither  expected  nor  appropriately  placed  in  a labora- 
tory; the  uniqueness  of  in-house  labs  can  be  satisfactorily  fulfilled  by 
operational  awareness."  Equally  important  is  awareness  of  the  technologi- 
cal goals  and  objectives  of  the  laboratory  involved.  The  basic  checklist 
talks  of  how  direction  and  policy  guidance  from  higher  echelons  is  dissemi- 
nated and  implemented.  It  questions  the  application  of  TNs  to  programs  and 

it  addresses  the  effective  distribution  of  technical  information  and  intelli- 
[32] 

gence.  The  missing  element  here,  though  one  could  possibly  argue  that 

it  is  implied,  is  the  examination  of  how  well  the  needs,  policies,  and  organi- 
zational goals  are  understood  by  all  personnel.  Robertson  notes  that  not  only 
must  management  begin  with  clearcut  goals  and  objectives,  it  must  insure  that 
they  are  clearly  understood  by  all  levels  in  the  organization.  The  in- 

spector must  assess  how  well  the  research  personnel  actually  do  understand 
these  needs  and  policy  objectives.  To  do  this  requires  the  inspector  to 
have  the  understanding  and  insight  I talked  of  earlier  in  this  paper. 

My  final  point  is  in  the  area  of  project  evaluation.  The  checklist 
asks  how  work  units  are  established,  how  the  research  program  is  evaluated 
and  what  criteria  are  used  for  establishing  priorities.  My  concern  here 
is  again  one  of  insufficient  emphasis,  for  it  may  be  said  that  "selection, 
evaluation,  and  termination  of  basic  research  projects  constitute  the  most 
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important  work  of  the  service  management  organizations,..."  If  this 

is  true,  then  some  of  the  important  aspects  of  the  evaluation  of  research 
projects  should  be  specifically  identified.  The  type  of  considerations 
that  characterize  a proper  evaluation  should  be  provided  to  guide  inspectors. 
Dean  and  Burgess  provide  useful  lists  that,  while  oriented  toward  industry, 
can  be  adapted  to  give  a valid  picture  of  the  appropriate  factors  to  be  con- 
sidered. ^2,5] 


SUMMARY 


In  this  paper,  I have  discussed  a perspective  on  how  the  application 
of  the  inspection  process  to  the  research  and  development  process  should  be 
conducted.  I have  discussed  the  need  for  improved  communication  between 
the  inspected  research  activities  and  the  inspection  teams,  and  I have 
recommended  specific  areas  of  investigation  that  could  enhance  the  meaning- 
fulness of  an  inspection.  The  paper  has  provided  food  for  thought,  but  only 
the  people  in  the  research  activities  and  the  people  in  inspection  can  make 
the  system  work  better.  It  is  up  to  them. 
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